Logit agents use the discrete choice model fit from the paper to pick which statistical area to trawl next.
More precisely, given a vector of logit coefficients \(\beta\) and the design matrix \(X\) where each row \(x_i\) contains the observations regarding statistical area \(i\), the probability of fishing in statistical area \(i\) is:
\[
\text{Pr}(i) = \frac{e^{x_i \beta}}{ \sum_{j} e^{x_j \beta}}
\]
Once a statistical area is picked, a cell at random within that area is trawled
Expressed in pseudo-R, logit agents act as follows:
# Given K statistical areas
# Logit coefficients beta
# And observations (distance, habit and intercepts) for each area
#compute utility (logits)
for(statistical_area in 1:K)
utility[statistical_area] = beta*observations[statistical_area,]
# Use SOFTMAX to pick one statistical area:
# probability of choosing arm i is proportional to the exp of its utility
denominator<- sum(exp(utility))
probabilities<- exp(utility)/denominator
# choose statistical area based on that probability
statistical_area<-sample(1:K,prob = probabilities,size = 1)
# target a random cell in that statistical area
target<-sample(cells[statistical_area],size=1)
This is the standard way to simulate historical behaviour in fishery agent-based models (see for example Saul, 2012; Rose et al., 2015). Because its \(\beta\) vector comes from a statistical fit, it requires no further calibration.
RUM agents function just like logit agents but:
For these agents the \(x\) vector in the logit probability is made up of the following:
There is a global intercept which is necessary for agents not to get stuck at the very first area they trawl.
There are two variants of the RUM agent: RUM fleetwide where the Revenue/hr \(x\) averages across the experience of the whole fleet rather than individual observations and RUM precise where the agents perform the logit selection over the finer POSEIDON cells rather than the statistical areas.
Historical agents are a less elegant (but more effective) way to force agents to act historically. Agents only consider statistical areas within 200km of their home port. They target each area with probability proportional to how often that area was visited between 2011 to 2014. For example, if the agent has to choose between statistical area A and B, and the logbook data shows that area A was visited 900 times while area B 100 times then the agent will visit A with probability \(\frac{9}{10}\).
Expressed more precisely in algorithm form:
# Given K statistical areas
# given trips[] vector containing empirically observed visits per statistical area
#ignore all areas that are too far
for(statistical_area in 1:K)
if(distance(statistical_area) > 200)
trips[statistical_area] = 0
# normalize trips into probabilities
trips<- trips/sum(trips)
# choose statistical area based on that probability
statistical_area<-sample(1:K,prob = trips,size = 1)
# target a random cell in that statistical area
target<-sample(cells[statistical_area],size=1)
If we judge statistical accuracy in terms of how much each statistical area was visited between 2011 and 2014, then the historical agent is more accurate than the logit one.
This is because logit agents have a high \(\beta\) associated with “habit” variable and tend to often get stuck in suboptimal areas.
When we judge statistical accuracy by random-utility choice fits, then the logit agents are more accurate than the historical ones.
Historical agents then look more accurate in aggregate but less so when studying their individual dynamics.
The multi-armed bandit problem (Bubeck & Cesa-Bianchi, 2012; Kuleshov & Precup, 2014; Vermorel & Mohri, 2005) is a classic model of the exploration-exploitation tradeoff.
In its original form an agent faces a set of slot machines; each slot machine’s payoff is stochastic but some machines are better than others. The agent can only play one slot machine at a time and has to balance the need to try new slot machines to learn their payoffs with the cost of not playing the slot machine that so far has proven superior.
Many heuristics, bandit algorithms, have been proposed to solve this class of problem but here we focus only on the simplest one: the \(\epsilon\) greedy bandit.
Each trip the agent has probability \(\epsilon\) of trying a statistical area at random. If not the agent will go back fishing the area whose average profits have been highest so far. At the end of each trip, the agent observes the profits made and updates its valuation of the statistical area visited. More precisely, the algorithm proceeds as follows:
# Given K statistical areas
# given experience[] vector containing average profits observed so far by the agent
# with probability epsilon, make random choice
if(runif() < epsilon)
chosen_area <- sample(1:K,size=1)
# otherwise pick the area that so far made the most profits
else
chosen_area<-which.max(experience)
target<-sample(cells[chosen_area],size=1)
# at the end of the trip, update your experience by
# exponential moving average
experience[chosen_area]<- alpha * experience[chosen_area] + (1-alpha) * profits_made
The algorithm has 2 parameters: the exploration rate \(\epsilon\) and the exponential moving average’s smoothing parameter \(\alpha\).
Explore-exploit-imitate is the default algorithm in POSEIDON.
Much like the \(\epsilon\)-greedy algorithm, it has a fixed probability of exploring a new cell (although in this instance the exploration is in the neighborhood of the last cell visited, not a random new statistical area).
When not exploring, the agent has a probability of imitating a “friend” who is making more money.
Otherwise the fisher targets the last cell trawled.
Each fisher has 2 “friends” she keeps track of. Each fisher knows where their friends last trip was and how much money it made. Friendship is only possible between boats of the same port. Friendships are not necessarilly reciprocal.
In pseudo-R the explore-exploit-imitate algorithm looks as follows:
# with probability epsilon, explore
if(runif()<epsilon)
# pick a new area at random in the neighborhood of your last cell visited
# where delta is the exploration range
options<- von_neumann_neighborhood(last_cell_visited,
delta)
#go there
target<-sample(options,size=1)
else
# if a friend is doing better, target their area with probability gamma
if(runif < gamma && profits[me] < profits[best_friend])
target<- position[best_friend]
# otherwise go back to the same cell
else
target<-last_cell_visited
#if explored and the profits are worse than before, then go back
if(profits(last_cell_visited)<profits(cell_visited_two_trips_ago))
last_cell_visited<-cell_visited_two_trips_ago
The algorithm depends on 3 parameters: the exploration rate \(\epsilon\), the exploration range \(\delta\) and the imitation probability \(\gamma\).
Heat-mapping agents build a statistical model of profit per cell (the eponymous heatmaps) and update it and the end of each trip. With a fixed probability \(1-\epsilon\) they target the peak of the heatmap otherwise fishing in its neighborhood.
More precisely the heat-mapping agent builds a statistical heatmap by using a kernel regression where for each cell \(x\) we initialize variables \(g_0,m_0=0\).
Whenever the fisher observes profits \(\Pi_y\) at cell \(y\), it updates its estimate profitability of cell \(x\) by:
\[
\begin{aligned}
g_n &= g_{n-1} + K(x,y) \\
m_n &= m_{n-1} + \frac{ \left(\Pi_y - m_{n-1} \right) K(x,y)}{g_n}
\end{aligned}
\]
Where \(K(\cdot,\cdot)\) is a kernel function and \(m_n\) is the predicted profitability of \(x\).
To give more weight to more recent observations we add a forgetting factor \(0<\lambda<1\) as follows:
\[
g_n = \lambda g_{n-1} + K(x,x_n)
\]
We use here as kernel function:
\[
K(x,y) = exp(-\frac{\text{Distance}(x,y)^2}{2\sigma})
\]
Where \(\sigma\) is the space bandwitdh (that is, it modulates how much far away observations affect profitability predictions here).
This recursive version of the Kernel regression first appeared in Krzyzak (1992).
Given the statistical model, the heat-mapping agents proceed as follows:
# given heatmap_predictions: the predictions made by the fisher about each cell profitability
best_cell<-which.max(heatmap_predictions)
# with probability epsilon, explore around the best_cell
if(runif()<epsilon)
# where delta is the exploration range
options<- von_neumann_neighborhood(best_cell,
delta)
#go there
target<-sample(options,size=1)
else
# pick the top
target<-best_cell
It is the strategy with more parameters to calibrate: \(\delta\) is the exploration range, \(\epsilon\) is the exploration probability, \(\sigma\) is the space bandwitdh, \(\lambda\) is the forgetting factor.
Random agent simply pick a cell at random each trip. It is a very poor strategy but it is important to use as a baseline: if other algorithms’ performance equals the random agents’ one then those algorithms aren’t useful. They have no parameters to calibrate
Perfect-SA agents pick a random cell for each statistical area, compute the profitability for each of those cells and pick one in proportion to the exponential of the expected profits (like logit agents). The agent has perfect knowledge in the sense that it knows ahead of time the profits she will make for each cell. However the agent does not update its computation halfway through a trip. This means that if multiple perfect agents target the same cell, their profits might be lower than expected at the beginning of the trip. These agents do not have the infinite computational power needed to analyse the game theoretical implications of everyone else being as informed as they do.
More precisely, perfect agents act as follows:
# given K statistical areas
#pick a cell for each area
for(statistical_area in 1:K)
cell[statistical_area]<-sample(cells[statistical_area],size=1)
#compute profits for each cell
profits[statistical_area]<-forecast_profits(cell)
# Use SOFTMAX to pick one statistical area
denominator<- sum(exp(profits))
# probability of choosing arm i is proportional to the exp of its utility
probabilities<- exp(profits)/denominator
# choose statistical area based on that probability
statistical_area<-sample(1:K,prob = probabilities,size = 1)
# target selected cell in that statistical area
target<-cell[statistical_area]
Perfect cell agents just pick the POSEIDON cell that makes the most profit knowing ahead of time how much fish is there in each cell. Like the Perfect SA agent it doesn’t take into consideration other fishers and their actions. Unlike Perfect SA it does not use logit to make its decision stochastic.
| Algorithm | Profit error | Hours out error | Sole error | Sablefish error | Long Thornyheads error | Short thornyheads error | Rockfish error | Error |
|---|---|---|---|---|---|---|---|---|
| Logit | 3.31 | 1.44 | 5.48 | 0.95 | 0.39 | 3.82 | 3.52 | 18.87 |
| Historical | 2.93 | 0.35 | 3.52 | 0.80 | 0.21 | 3.00 | 3.52 | 14.42 |
| RUM - precise | 4.24 | 0.78 | 5.76 | 0.77 | 2.42 | 5.25 | 3.52 | 22.71 |
| RUM | 2.36 | 0.88 | 3.75 | 0.24 | 0.60 | 3.42 | 3.52 | 14.97 |
| RUM Fleetwide | 3.25 | 2.86 | 5.04 | 1.51 | 0.39 | 4.27 | 3.52 | 21.08 |
| Perfect - SA | 0.38 | 1.63 | 2.44 | 0.28 | 1.35 | 3.66 | 35.20 | 45.09 |
| Perfect - Cell | 1.09 | 1.75 | 5.39 | 0.81 | 1.44 | 1.68 | 40.94 | 52.27 |
| Random | 6.30 | 6.64 | 9.76 | 14.02 | 7.17 | 9.78 | 3.52 | 57.19 |
| Bandit | 3.13 | 1.51 | 4.49 | 1.54 | 0.63 | 4.04 | 12.74 | 28.43 |
| EEI | 0.80 | 0.89 | 1.03 | 0.38 | 1.23 | 2.11 | 3.52 | 10.37 |
| EEI - Uncalibrated | 0.36 | 1.79 | 2.59 | 0.75 | 0.79 | 1.77 | 3.52 | 11.80 |
| Heatmap | 1.70 | 0.46 | 2.03 | 0.17 | 1.23 | 2.03 | 3.52 | 11.19 |
| Social Annealing | 5.75 | 6.88 | 6.92 | 8.73 | 5.38 | 7.71 | 3.52 | 45.51 |
Holland (2016) tabulates the quota prices between 2011 and 2014 observed in the groundfish fishery. We can compare this data with the quota prices generated from our model. Holland however reports that a large amount of quotas are bartered rather than traded and that prices vary much during the year: only 18% of quota transfers in 2011 involved cash.
We look at the price of quotas traded within the simulation and compare them with Holland’s numbers. Fundamentally, the constraining species in the model is sablefish whose quota is most expensive; the better the algorithm is at fishing, the more sablefish ends up costing. Other quotas are rarely traded and if so they are at the minimum price (0.05); this is true also for yelloweye which is so rare that is simply not traded as nobody really form high expectations of catching them.
Figure 2.1: Five sets of box plots, one for each species’ quota prices. The box plots show the yearly average trade price over 100 simulated runs. The red line is the 2011-2014 average as in table 2 of Holland (2016)
As a second test we plot average daily profits over the years in figure 2.2. Data on profit trends from the FISHeries Economics Explorer dataset (Northwest Fisheries Science Center, 2017) shows a peak in variable costs net revenue in 2011, then a slight decline until another peak in 2015. However the confidence intervals on the time series (with standard deviation approximately equal to the mean) are much larger than the trend itself which make any inference and calibration suspicious. Our main focus here was to make sure there was movement of more than 2 or 3 times the mean itself.
Figure 2.2: The simulated profits time series, one for each adaptive decision-making algorithm
As a third test we plot the number of fishers left in the simulation at the end of each year in figure 2.3. From the catcher vessel report we know the number of fishers remaining in the DTS fishery (Steiner, Pfeiffer, Harley, Guldin, & Lee, 2016), however these numbers change between each revision of the dataset. Moreover each year about 10-15% of boats in the West-Coast groundfish fishery make operating losses so that our simulated quitting decision (making two consecutive years of losses) might be overly stringent. Still, most simulations settle around the right number of fishers.
Figure 2.3: Number of active fishers left in each simulation each year. The dashed red line shows the number of non-whiting DTS boats left by 2015.
Finally, we plot in figure 2.4 the active fishers per port at the beginning and at the end of the simulation when using explore-exploit-imitate agents. The geographical allocation of other adaptive algorithms is similar. The model tends to predict too few fishers in Astoria and too many in Newport and Coo’s Bay. The model however does predict the decrease in DTS participation in Washington and is also accurate in California.
Figure 2.4: Box-plot of active fishers per port (ordered from north to south along the coast) when the decision-making algorithm is the calibrated explore-exploit-imitate over 100 simulated runs. The red squares in the 2014 plot are the active fishers per port reported by Steiner et al. (2016)
Because our model runs only for a short amount of time, we incur the risk of calibrating the model too tightly around the few years for which we have data. In this section we test for this risk by starting the model in 2007 rather than 2011. This involves changing starting abundance for each species, different gas prices, variable costs and implementing the 2011 regulatory shock when a different set of quotas were introduced and made tradeable. We run the model again until 2015 without recalibrating it. We test its prediction of both the 2011-2014 period (calibration error) and 2015 (validation error).
Before 2011, the fishery was regulated through bi-monthly trip limits (National Marine Fisheries Service, 2009). These rules changed frequently, were sometimes superseded by state regulations and varied between states. We did not try to model these accurately as we used no pre-2011 data to calibrate or validate the model. Our main focus was to have a different regime policy our agents would transition away from.
We follow again Toft, Punt, & Little (2011) here in simplifying these regulations as individual non-tradeable quotas. We need to do so because we do not simulate explicitly fishing seasons and weather patterns. Table 3.1 shows the aggregate quota numbers we derived. All quotas are higher except for Dover sole; however because Dover sole attainment rate is low, the overall effect of using pre-2011 quotas is more total landings.
| Species | Yearly Quota (mt) |
|---|---|
| Dover Sole | 16,866 |
| Sablefish | 3,457.8 |
| Shortspine | 1,927.8 |
| Longspine | 2,577.6 |
| Yelloweye | 67.1 |
Figure 3.1: Box plot showing the fishing outcomes error for 100 simulations of each decision algorithm. All runs started in 2007 and transitioned to ITQ in 2011
All the results from validation are upheld. Adaptive agents do better, statistical agents underperform in 2015, perfect agents catch too much yelloweye and finally bandit and social annealers fit worse than in the calibration phase.
The main difference is that heatmap agents now do better than EEI.
In this section we change the distribution of fish while keeping the overall number and stock assessment-driven dynamics constant. We show that the model is sensitive to this change and that the main driver is the degree of smoothness by which fish is initially distributed.
Originally we distributed fish according to the Essential Fish Habitat statistical models; here we distribute fish proportional to the CPUE (catches per unit of effort) recorded in the logbook data for each statistical area.
Branch et al. (2006) provides plenty of reasons not to use CPUE as an abundance metric and we only do so for sensitivity analysis.
Logbook observations are sparse so we apply a linear interpolator to have a CPUE prediction for each cell of the map.
The next figure shows how the abundance of sablefish changes as we change data source. Because of the linear interpolation, the CPUE-derived map distributes fish much more smoothly.
Figure 3.2: Image capture of abundance for sablefish in the model as derived from the EFH dataset (on the left) and the CPUE interpolation (on the right)
Figure 3.3 shows the errors obtained for each decision-making algorithm when calibrated against the new distribution of fish. Three results emerge. First, the error in absolute terms is much higher for all algorithms. Second, statistical algorithms outperform adaptive algorithms. Third, perfect agents perform as well as adaptive algorithms. The only result that survives this sensitivity analysis is that random agents still perform very poorly.
Figure 3.3: The calibration errors when using the CPUE map, each box plot represents 100 simulation runs.
All these results are due to the smoothness of geographical abundance. Figure 3.4 shows the profit simulated for each decision-making algorithm. The figure makes clear that it is not the profit maximizer agent that got more realistic, it is adaptive agents that now also make too much money. Smooth abundance distribution generates smooth profit curves that are easy for all adaptive agents to climb. When the problem is easy, assuming adaptive agents is like assuming perfect agents. Statistical agents perform well because they do not take advantange of the easier problem: they are forced to repeat historical choices even if better alternatives are present.
Figure 3.4: Yearly profits made by each decision-making algorithm in 100 simulated runs. Red dashed line is the empirical 2011-2014 average
It is instructive to look in detail at the consequences of using CPUE-driven maps. One subtle property of the map is shortspine and sablefish being almost perfectly co-located. This has less to do with the two species being actually co-located and instead simply represents the efficiency of fishermen in Astoria (a good reason for not using CPUE-driven maps). The model reacts in a realistic way: because sablefish is the target species and it is now impossible landing it without catching shortspine as well, shortspine quotas become more expensive; as we show in figure 3.5 quotas are sometimes even traded at prices higher than shortspine’s worth, meaning that agents take a loss by landing shortspine just so that they can land sablefish as well.
Figure 3.5: Shortspine thornyhead quota prices traded for each simulation using either EFH or CPUE abundance maps. The red dashed line represents the empirical quota price observed, the dashed black line represents the ex-vessel price. With the CPUE-driven map we see not only quotas more expensive than what was empirically observed but sometimes more expensive than their sale prices
It is not possible to speculate whether CPUE maps are always a poor idea for agent-based models. In this fishery effort is clustered near the coast and around some key Oregon ports, while the EFH allocates fish further out and more homogeneously. The issues with linear interpolations can also be prevented by higher resolution observations.
One problem with computing profits by averaging over boats is that there are a few agents who only make one trip each season filling only a limited amount of quota. The profits made by those boats are small and they tend to lower the average profit made by the fishery.
We test this here by recalibrating the model to observe only profits of boats making more than one trip a season.
This shuffles the ranking of adaptive agents as now heatmap agents do better than EEI, keeping most of the rest of the results constant.
The one major difference is that ranking by elastic nets in this instance prefers historical agents to adaptive ones.
| Algorithm | Profit error | Hours out error | Sole error | Sablefish error | Long Thornyheads error | Short thornyheads error | Rockfish error | Error |
|---|---|---|---|---|---|---|---|---|
| Logit | 1.8294189 | 1.5956562 | 5.520635 | 0.9186937 | 0.6479871 | 3.749891 | 3.515342 | 17.852234 |
| Historical | 0.3490466 | 0.3777594 | 3.570956 | 0.7183515 | 0.2681753 | 2.694667 | 3.515342 | 11.568638 |
| RUM - precise | 8.8917897 | 5.3729929 | 5.844995 | 2.3931682 | 0.9290197 | 4.156156 | 3.515342 | 31.444476 |
| RUM | 3.6251249 | 0.9467573 | 4.191578 | 0.3432829 | 0.8596639 | 3.402410 | 3.515342 | 17.247671 |
| RUM Fleetwide | 4.6833798 | 1.8037876 | 3.933527 | 0.8442448 | 0.4295544 | 3.112559 | 3.515342 | 18.095230 |
| Perfect - SA | 1.9526730 | 1.6446654 | 2.140072 | 0.3400391 | 0.9314792 | 3.599848 | 42.114346 | 52.399944 |
| Perfect - Cell | 3.0637555 | 1.9553023 | 4.834326 | 0.7138970 | 1.2262521 | 1.654332 | 41.708547 | 54.695846 |
| Random | 6.3009470 | 6.6408586 | 9.758838 | 14.0225794 | 7.1675299 | 9.781408 | 3.515342 | 57.187502 |
| Bandit | 2.3074316 | 1.4537440 | 4.592497 | 1.5115470 | 0.3439748 | 3.679348 | 13.584397 | 27.472469 |
| EEI | 3.9981258 | 0.9409031 | 1.041330 | 0.3598488 | 1.0175235 | 2.225416 | 3.515342 | 14.258213 |
| EEI - Uncalibrated | 2.7000278 | 1.8156970 | 2.451744 | 0.6749145 | 0.5993801 | 1.689115 | 3.515342 | 14.038406 |
| Heatmap | 0.6822292 | 0.5493656 | 1.906921 | 0.1646146 | 1.0003418 | 1.597071 | 3.515342 | 9.413672 |
| Social Annealing | 2.8436309 | 7.0646494 | 6.999442 | 8.9278768 | 5.3837957 | 7.769929 | 3.515342 | 43.201325 |
| name | p_value |
|---|---|
| Logit | <2e-16 |
| Historical | <2e-16 |
| RUM - precise | <2e-16 |
| RUM | <2e-16 |
| RUM Fleetwide | <2e-16 |
| Perfect - SA | <2e-16 |
| Perfect - Cell | <2e-16 |
| Random | <2e-16 |
| Bandit | <2e-16 |
| EEI | <2e-16 |
| EEI - Uncalibrated | <2e-16 |
| Heatmap | 1 |
| Social Annealing | <2e-16 |
| Bandit | EEI | EEI - Uncalibrated | Heatmap | Historical | Logit | Perfect - Cell | Perfect - SA | Random | RUM | RUM - precise | RUM Fleetwide | Social Annealing | method |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.198 | 0.046 | 0.076 | 0.404 | 0.030 | 0.030 | 0.000 | 0.000 | 0 | 0.023 | 0.109 | 0.084 | 0.000 | ABC |
| 0.026 | 0.270 | 0.120 | 0.398 | 0.144 | 0.004 | 0.006 | 0.012 | 0 | 0.002 | 0.000 | 0.010 | 0.008 | RF |
| 0.000 | 0.001 | 0.001 | 0.110 | 0.888 | 0.000 | 0.000 | 0.000 | 0 | 0.000 | 0.000 | 0.000 | 0.000 | Elastic Nets |
In the model we remove quotas that eventually were landed by other gears. This is accurate but had we truly looked into the future rather than hindcasting we would have not had that information. Would behavioural ranking change if we made north and south sablefish quota all available to the DTS boats? Behavioural ranking does not change but errors are in general all higher due to too much sablefish being landed.
| Algorithm | Profit error | Hours out error | Sole error | Sablefish error | Long Thornyheads error | Short thornyheads error | Rockfish error | Error |
|---|---|---|---|---|---|---|---|---|
| Logit | 2.5062546 | 0.4849709 | 4.554425 | 4.836151 | 1.8638140 | 1.8752563 | 3.515342 | 19.64193 |
| Historical | 2.7194738 | 0.2270279 | 2.718053 | 4.644004 | 1.5580409 | 2.0805770 | 3.515342 | 17.53282 |
| RUM - precise | 4.8863993 | 4.2127717 | 3.961964 | 2.061245 | 0.6287253 | 3.1941339 | 3.515342 | 22.52637 |
| RUM | 1.5524370 | 0.9696508 | 2.745559 | 4.696463 | 0.9356159 | 1.9838542 | 3.515342 | 17.68363 |
| RUM Fleetwide | 2.4288213 | 4.0043633 | 4.237415 | 3.555068 | 0.6268260 | 3.2946209 | 3.515342 | 21.49824 |
| Perfect - SA | 0.8847935 | 1.1453905 | 3.048719 | 5.334109 | 0.3337529 | 1.4312787 | 34.653870 | 46.78696 |
| Perfect - Cell | 2.1576156 | 0.4324112 | 7.195604 | 5.355205 | 0.8201577 | 0.8598929 | 3.515342 | 20.34080 |
| Random | 6.3009470 | 6.6408586 | 9.758838 | 8.265836 | 7.1675299 | 9.7814078 | 3.515342 | 51.43076 |
| Bandit | 3.0405715 | 5.3619350 | 3.956688 | 2.418132 | 0.5785757 | 3.5733526 | 6.654128 | 25.91516 |
| EEI | 0.5251665 | 0.6308443 | 1.325388 | 4.766255 | 0.7976880 | 2.4215684 | 3.515342 | 14.29459 |
| EEI - Uncalibrated | 0.9760839 | 0.5903257 | 2.793881 | 5.373538 | 0.6654905 | 1.4480855 | 3.515342 | 15.55882 |
| Heatmap | 2.2097483 | 4.4247498 | 2.028676 | 3.506410 | 0.3423249 | 1.7497237 | 3.515342 | 17.73465 |
| Social Annealing | 5.9848003 | 8.8240227 | 8.366549 | 6.106372 | 6.1007978 | 8.6185749 | 3.515342 | 47.30244 |
| Bandit | EEI | EEI - Uncalibrated | Heatmap | Historical | Logit | Perfect - Cell | Perfect - SA | Random | RUM | RUM - precise | RUM Fleetwide | Social Annealing | method |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.191 | 0.087 | 0.037 | 0.372 | 0 | 0.044 | 0 | 0.000 | 0 | 0.073 | 0.150 | 0.045 | 0.000 | ABC |
| 0.010 | 0.626 | 0.140 | 0.010 | 0 | 0.000 | 0 | 0.004 | 0 | 0.034 | 0.006 | 0.144 | 0.026 | RF |
| 0.000 | 0.998 | 0.000 | 0.000 | 0 | 0.000 | 0 | 0.000 | 0 | 0.002 | 0.000 | 0.000 | 0.000 | Elastic Nets |
Miller (1998) introduces active non-linear tests as a search problem (see chapters 3-5 of Ligeza, 1995): look in the neighborhood of the optimal parameters for the worst possible outcome.
In its original implementation a search around 10% neighborhood of calibrated parameters for the WORLD3 model (Meadows, Meadows, Randers, & Behrens, 1972) delivered predictions completely at odds with the authors’ original claims.
Stonedahl & Wilensky (2010) provides another example searching in the 10% neighborhood of the Anasazi model (see also Lee et al., 2015).
Here we use active non-linear tests to study the sensitivity of our calibrated parameters.
We first perform an active non-linear test to the calibrated catchability (\(Q\)) and hold-size (\(H\)) parameters by varying them 10% around their optimal value with historical agents. We show the density plot of fishing outcome error in figure 4.1. The search increases the error by only 11% on average. While the difference is significant to a Wilcox test, it is linearly proportional to the 10% shock in parameters.
Figure 4.1: The histogram of fishing outcome errors generated by 100 runs of the step 1 calibrated model (that is, using historical agents) at the bottom versus the worst performing set of parameters found by the active non-linear test.
Explore-exploit-imitate agents are very insensitive to parameter changes. In figure 4.2 we compare the calibrated and worst possible parameter sets (changing behavioural parameters by at most 10%). The error is statistically significant to a Wilcoxon test, but the overall increase is only of 6%. Even though the explore-exploit-imitate sensitivity analysis has more parameters it is actually stabler the historical agents used above.
Figure 4.2: The density plot of fishing outcome errors generated by 100 runs of the step 2 calibrated model using explore-exploit-imitate at the bottom versus the worst performing set of parameters found by the active non-linear test.
Heat-mapping agents are more sensitive to active non-linear tests (particularly to changes in forgetting factor \(\lambda\)). As shown in figure 4.3 the average fishing outcome error increases only by 13% but the likelihood of large errors is higher. In a way this is stating the obvious: heat-mapping agents are more complex than explore-exploit-imitate ones and we thus expect them to be more delicate to parameter changes.
Figure 4.3: The density plot of fishing outcome errors generated by 100 runs of the step 2 calibrated model using heatmap agents at the bottom versus the worst performing set of parameters found by the active non-linear test.
From Alaskan tagging data we know that sablefish moves on average 191 km per year (Hanselman, Heifetz, Echave, Dressel, & Jech, 2015). We also know from simple abstract model runs that fish diffusion rates in POSEIDON have large effects on long term profitability and behaviour of the simulated fleet. For these reasons we initially classed sablefish movement as a free parameter to calibrate much like catchabilities and hold-sizes. In these early calibration runs however movement rate did not have a significant effect in minimizing distance from data (which in a Bayesian optimization can be noticed by the very large bandwitdhs assigned to the variable by the optimiser or just by plotting posteriors). In the final version of the calibration presented above the movement rate was simply set to 0 as it made the model much faster to run.
Similar to Goethel et al. (2011), we modeled movement as a simple diffusion rate between cells. However, we biased movement in such a way that it would never alter the patchiness of the original map. Movement in this simulation is not from cells where there are more fish to cells where there are a few; rather fish moves from “full” areas to “empty” ones where full and empty are defined in terms initial abundance distribution. To make a concrete example, take cell A and a contiguous cell B where cell A contained \(k_a\) percent of the total sablefish at the beginning of the simulation and cell B contained \(k_b\) percent. If there currently are \(f_a\) sablefish in cell A and \(f_b\) in cell B, then fish movement between A and B is given by: \[ m \frac{f_a k_b - f_b k_a}{k_a+k_b} \] where \(m \in [0,1]\) is the diffusion rate parameter. This means that even at maximum movement rate distribution of fish remains as patchy.
Figure 4.4 shows the difference in error generated by maximizing sablefish diffusion rate. Validation error is fundamentally unchanged. Calibration error (not shown) increases by 9%. While we know that POSEIDON is sensitive to movement rates in other scenarios here we showed that when simulating short periods of time there is little error added by ignoring movement and it is anyway hard to back out a proper movement rate from data without longer observations.
Figure 4.4: Density plot of validation error generated by 100 runs with movement turned off and with movement turned on, \(m=0.1\)
RUM agents perform poorly in the validation section primarily by overfitting. Would they have been much different if we had “cheated” and calibrated them against validation error? Statistical area agents (RUM and RUM SA) see almost no improvement but RUM precise agents can halve their validation error if reparametrized. Figure 5.1 shows the difference in calibration and validation error distribution for the original RUM precise agent (whose parameters minimize calibration error) and the “cheating” RUM precise agent (whose parameters minimize validation error).
Figure 5.1: Empirical densities of validation and calibration error for RUM precise agents. As expected the cheating agent sacrifices fit quality in the calibration phase in order to do better in validation.
The key result of this exercise is to compare the optimal parameters for the original RUM precise and the one that minimizes validation error instead:
| Parameter | Original | Validation Minimizer |
|---|---|---|
| Dover sole CPUE | -0.5112 | -0.0773 |
| Sablefish CPUE | -0.0086 | 0.55 |
| Yelloweye CPUE | -0.345 | 0.714 |
| Distance | -0.0074 | -0.01 |
| Habit | 2.4855 | -2.5425 |
| Revenue | 0.6633 | 0.4148 |
| Intercept | 45.58 | 44.07 |
The main change is the difference in habit which goes from very positive to very negative (compare this to the 0.21 coefficient in the logbook-derived Logit agent). This is introducing purposeful exploration “by the backdoor”: it is a way to keep the agent changing its destination even if the other indicators do not change.
This exploration makes a difference in even the few years that are simulated as figure 5.2 shows: profits decrease much faster for the original RUM Precise than the cheating one which are pushed to move around over time.
Figure 5.2: Empirical densities of validation and calibration error for RUM precise agents. As expected the cheating agent sacrifices fit quality in the calibration phase in order to do better in validation.
Beecham, J. A., & Engelhard, G. H. (2007). Ideal free distribution or dynamic game? An agent-based simulation study of trawling strategies with varying information. Physica A: Statistical Mechanics and Its Applications, 384(2), 628–646. https://doi.org/10.1016/j.physa.2007.05.053
Branch, T. A., Hilborn, R., Haynie, A. C., Fay, G., Flynn, L., Griffiths, J., … Young, M. (2006). Fleet dynamics and fishermen behavior: lessons for fisheries managers. Canadian Journal of Fisheries and Aquatic Sciences, 63(7), 1647–1668. https://doi.org/10.1139/f06-072
Bubeck, S., & Cesa-Bianchi, N. (2012). Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. https://doi.org/10.1561/2200000024
Hanselman, D. H., Heifetz, J., Echave, K. B., Dressel, S. C., & Jech, J. M. (2015). Move it or lose it: movement and mortality of sablefish tagged in Alaska. Canadian Journal of Fisheries and Aquatic Sciences, 72(2), 238–251. https://doi.org/10.1139/cjfas-2014-0251
Holland, D. S. (2016). Development of the Pacific Groundfish Trawl IFQ Market. Marine Resource Economics, 31(4), 453–464. https://doi.org/10.1086/687829
Krzyzak, A. (1992). Global Convergence of the Recursive Kernel Regression Estimates with Applications in Classification and Nonlinear System Estimation. IEEE Transactions on Information Theory, 38(4), 1323–1338. https://doi.org/10.1109/18.144711
Kuleshov, V., & Precup, D. (2014). Algorithms for multi-armed bandit problems. https://doi.org/10.1145/1109557.1109659
Lee, J. S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., … Parker, D. C. (2015). The complexities of agent-based modeling output analysis. Journal of Artificial Societies and Social Simulation, 18(4). https://doi.org/10.18564/jasss.2897
Ligeza, A. (1995). Artificial Intelligence: A Modern Approach (Vol. 9, pp. 215–218). Prentice Hall. https://doi.org/10.1016/0925-2312(95)90020-9
Meadows, D. H., Meadows, D. L., Randers, J., & Behrens, W. W. (1972). The limits to growth. New York, 102, 27.
Miller, J. H. (1998). Active Nonlinear Tests (ANTs) of Complex Simulation Models. Management Science, 44(6), 820–830. https://doi.org/10.1287/mnsc.44.6.820
National Marine Fisheries Service. (2009). Pacific Coast Groundfish Fishery, Trip limit Tables for January 1, 2010. Seattle, WA: NMFS.
Northwest Fisheries Science Center. (2017). FISHeries Economics Explorer (FISHEye). Retrieved from https://dataexplorer.northwestscience.fisheries.noaa.gov/fisheye/
Rose, K. A., Fiechter, J., Curchitser, E. N., Hedstrom, K., Bernal, M., Creekmore, S., … Megrey, B. A. (2015). Demonstration of a fully-coupled end-to-end model for small pelagic fish using sardine and anchovy in the California Current. Progress in Oceanography, 138, 348–380.
Saul, S. E. (2012). An Individual-based Model to Evaluate the Effect of Fisher Behavior on Reef Fish Catch Per Unit Effort.
Steiner, E., Pfeiffer, L., Harley, A., Guldin, M., & Lee, T. (2016). Economic Data Collection Program Catcher Vessel Report (2009–2014). Seattle, WA: NOAA NWFSC.
Stonedahl, F., & Wilensky, U. (2010). Evolutionary Robustness Checking in the Artificial Anasazi Model. In AAAI fall symposium: Complex adaptive systems (pp. 120–129).
Toft, J. E., Punt, A. E., & Little, L. R. (2011). Modelling the economic and ecological impacts of the transition to individual transferable quotas in the multispecies US west coast groundfish trawl fleet. ICES Journal of Marine Science, 68(7), 1566–1579. https://doi.org/10.1093/icesjms/fsr095
Vermorel, J., & Mohri, M. (2005). Multi-Armed Bandit Algorithms and Empirical Evaluation BT - Machine Learning: ECML 2005. In Machine learning: ECML 2005 (pp. 437–448). Springer. https://doi.org/10.1007/11564096_42
1.6 Social annealing agents
Social annealing agents are a slight variation over EEI where they do not know where other fishers trawl but they do know the profits others are making. Whenever a social annealing agent is making \(k\)% worse than average fishery profits, she explores a new cell at random. Otherwise she targets to the last cell visited.
More precisely, the algorithm proceeds as follows:
The algorithm depends on two parameters: the threshold parameter \(k\) and the exploration range (in map cells) \(\delta\). This algorithm first appeared (with \(\delta=\infty\)), to the best of our knowledge, in Beecham & Engelhard (2007).